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EXECUTIVE  SUMMARY 


A.  PURPOSE 

The  purpose  of  this  document  is  to  describe  the  combat  outcomes  of  the 
Countermine  Experiment  (CME),  a  Distributed  Interactive  Simulation  (DIS)  exercise 
conducted  at  the  Mounted  Warfare  Testbed  at  Ft.  Knox,  KY,  in  July  1996.  This  analysis 
is  based  on  the  fact  that  the  CME  was  structured  on  a  2^  factorial  design,  which  suggests 
a  specific  linear  model  as  the  predictor  of  experimental  outcomes.  Classical  analysis  of 
variance  is  used  to  determine  the  significant  factors  in  this  model.  These  factors  are 
analyzed  in  terms  of  the  context  of  the  experiment. 

The  CME  was  conducted  under  the  direction  of  the  Night  Vision  Electronic 
Sensors  Division,  Ft.  Belvoir,  VA,  and  the  Engineer  Battle  Testbed,  Ft.  Leonard  Wood, 
MO.  It  entailed  manned  and  unmanned  simulators  in  a  company-level  armored 
sngagement  in  which  antitank  mines  and  countermine  equipment  played  a  fundamental 
role.  The  purpose  of  this  test  was  to  gain  insight  into  the  potential  utility  of  new  land 
mine  detection  and  neutralization  technologies. 

A  shorter  and  more  elementary  version  of  the  document  was  prepared  as  a 
working  paper  under  task  T-Dl-1230. 

B.  OVERVIEW  OF  CME  TESTS 
1.  Technologies 

Three  innovative  countermine  technologies  were  explored  in  CME:  the  Airborne 
Standoff  Minefield  Detection  System  (ASTAMIDS),  the  Ground  Standoff  Minefield 
Detection  System  (GSTAMIDS),  and  the  Explosive  Standoff  Minefield  Breacher 
(ESMB).  The  first,  ASTAMIDS,  is  a  sensor-laden  Unmanned  Aerial  Vehicle  (UAV) 
designed  to  detect  land  mines  emplaced  in  conventional  string,  lattice,  or  scatter  patterns. 
Two  versions  of  this  system  are  under  consideration:  one  with  a  passive  IR  sensor,  and 
another  with  an  active  laser  and  a  passive  IR  sensor.  Sensor  data  are  transmitted  from  the 
UAV  to  a  mobile  ground  station  for  processing.  ASTAMIDS  flies  at  80  knots  at  an 
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altitude  between  100  and  300  meters.  Its  sensor  footprint  is  65  meters  wide  (at  an  altitude 
of  100  meters).  This  system  was  modeled  with  a  75%  probability  of  detecting  a  given 
antitank  (AT)  mine  swept  by  its  sensor. 

The  second  technology  is  the  Ground  Standoff  Minefield  Detection  System.  This 
is  an  experimental  system  consisting  of  a  configuration  of  ground  penetrating  radar,  along 
with  ultraviolet  and  X-ray  backscatter  detection  sensors.  It  is  designed  to  detect  both 
magnetic  and  nonmagnetic  antitank  mines.  In  CME,  GSTAMIDS  was  modeled  with  a  40 
meter  standoff  range  and  a  40  x  60  square  meter  resolution  cell.  The  probability  of 
detection  of  an  AT  mine  within  a  cell  was  75%;  however,  three  or  more  mines  needed  to 
be  detected  before  GSTAMIDS  reported  that  a  cell  contained  mines. 

The  third  and  final  technology  explored  in  CME  was  the  Explosive  Standoff 
Minefield  Breacher  (ESMB).  This  system  consists  of  a  rocket-ferried  5  x  145  square 
meter  net  containing  shape  charges  capable  of  neutralizing  surface  and  buried  land  mines. 
The  ESMB  was  mounted  on  a  trailer  and  towed  by  an  Ml  A1  and  had  a  standoff  range  of 
45  meters  in  CME.  As  modeled,  the  system  had  a  95%  probability  of  neutralizing  any 
mine  covered  by  its  net. 

2.  Trial  Conduct 

The  CME  consisted  of  32  trials,  conducted  over  a  4-week  period,  in  which  each 
possible  combination  (eight  in  all)  of  the  three  technologies  was  tested  four  times.  The 
four  replications  of  each  mix  of  countermine  equipment  were  run  on  distinct  terrain 
tracts,  or  regions,  of  the  Fort  Knox  Military  Reservation  (a  simulated  terrain  database). 
Formally,  this  is  an  example  of  a  2^  factorial  experimental  design  in  which  three  factors 
are  each  tested  at  two  levels  with  four  repeated  measurements.  Each  technology  appeared 
symmetrically  in  half  the  trials.  For  example,  half  had  ASTAMIDS  data  (collected  in  a 
preliminary  phase  of  the  experiment),  half  did  not.  One  quarter  had  both  ASTAMIDS 
data  and  GSTAMIDS;  one  quarter  had  neither;  one  eighth  had  all  three  technologies.  In 
trials  without  ASTAMIDS  data.  Blue  received  no  aerial  detection  intelligence.  In  trials 
without  GSTAMIDS,  Blue  had  no  ground  detection  system.  In  trials  without  ESMB, 
Blue  had  access  to  a  conventional  breaching  system,  the  Armored  Vehicle  Launched 
MICLIC  (AVLM).  In  all  trials.  Blue  had  use  of  tank  mounted  plows  and  rollers  for  mine 
clearing  operations. 

In  each  trial,  the  Blue  force  consisted  of  one  MlAl  armored  company,  an 
Ml  13A2  engineering  platoon,  and  an  additional  MlAl  platform  for  the  breaching  system 
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or  ground  surveillance  system  or  both.  The  armored  company  commander’s  MlAl  and 
one  vehicle  from  each  Blue  platoon  (the  platoon  leader’s)  were  manned;  all  the  others 
were  Modular  Semi- Automated  Force  (ModSAF)  entities.  Five  of  the  unmanned  MlAls 
were  outfitted  with  either  plows  or  rollers.  In  addition,  Blue  was  supported  by  an  M107 
artillery  unit.  OPFOR  consisted  of  two  ModSAF  BMP2s  and  one  T80.  OPFOR  artillery 
was  represented  by  the  ModSAF  “Bomb  Button,”  a  mechanism  through  which  an 
operator  can  deliver  ordnance  without  a  tube  representation.  Also,  between  four  and  six 
200  m  X  100  m  minefields,  each  containing  140  antitank  mines,  were  deployed  in  each 
trial. 

The  objective  in  each  trial  was  determined  by  a  Blue  battalion  commander.  This 
was  conveyed  to  the  Blue  company  commander  in  a  series  of  OP-ORDERs  and  FRAGOs. 
The  company  commander,  in  turn,  conveyed  orders  to  his  unit  in  pre-trial  briefings. 
Trials  were  conducted  under  the  supervision  of  the  Blue  battalion  commander.  In 
particular,  he  decided  when  the  objective  was  attained  and  when  each  trial  was  complete. 

Data  were  recorded  by  ten  research  assistants,  video  tapes,  the  battalion 
commander,  ModSAF  plan  view  imagery,  and — ^most  relevant  to  this  paper — electronic 
loggers  that  captured  DIS  protocol  data  units.  In  addition,  after-action  reviews  and 
weekly  summary  sessions  provided  “feed  back”  and  subjective  insight. 

C.  SUMMARY  OF  RESULTS  AND  CONCLUSIONS 

This  paper  analyzes  three  measures:  Blue  losses  to  mines,  to  direct  and  indirect 
fire  combined,  and  losses  to  all  agents  combined.  A  loss  in  this  context  means  any  level 
of  destruction:  catastrophic,  mobility,  or  firepower  (without  double  counting,  of  course). 
Of  specific  interest  is  the  impact  of  the  various  mixes  of  countermine  equipment  on  these 
losses.  In  particular,  an  attempt  will  be  made  to  determine  the  most  significant  factors 
affecting  (reducing  or  increasing)  Blue  losses  by  computing  analysis-of-variance 
(ANOVA)  tables  for  the  main  factors  in  this  test  and  their  interactions.  Little  attention 
will  be  paid  to  OPFOR  losses  as  there  were  only  three  OPFOR  vehicles  played  in  each 
trial.  Generally — ^with  only  two  exceptions— either  two  or  three  of  these  were  destroyed 
(average  OPFOR  loss  was  2.5  vehicles). 

This  analysis  differs  from  IDA  Paper  P-3300,  “Analysis  of  an  Army  Countermine 
Top  Level  Demonstration  -  The  Countermine  Experiment,”  in  several  ways.  First,  in  the 
present  document,  losses  to  direct  and  indirect  fire  are  analyzed  explicitly.  Secondly,  the 
impact  of  scenarios  is  examined  (as  a  co variate  to  the  basic  factorial  design).  Finally,  the 
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earlier  IDA  Paper  includes  an  analysis  of  temporal  measures,  whereas,  in  this  document, 
attention  is  confined  to  vehicular  losses. 

Figure  1  is  borrowed  from  the  main  body  of  this  report  and  summarizes  Blue 
losses  for  each  combination  of  countermine  equipment.  It  gives  some  indication  of  the 
impact  of  ASTAMIDS  on  reducing  Blue  losses,  but  also  suggests  some  “uneven” 
behavior  among  the  various  countermine  systems.  For  example,  the  reduction  in  losses  to 
mines  when  AVLM  and  ASTAMIDS  data  are  used  in  conjunction  does  not  carry  through 
to  the  cases  in  which  ESMB  and  ASTAMIDS  are  used  together. 


Total  Kills  of  Blue  Vehicles  by  Agent 


nlylF." . 

■  By  D.F. 

■  By  Mines 


BASE  ESMB  GST  COMBO 


CM  Configuration 


Figure  1.  Blue  Vehicle  Losses  by  Agent 

Since  the  CME  was  conducted  as  a  classical  factorial  experiment,  the  expected 
outcomes  displayed  in  Figure  1  are  determined  by  a  specific  linear  model.  In  the  case  of 
a  2^  factorial,  that  model  is 

E(yijkn)  =  u  +  ai  +  bj  +  gk  +  abij  +  bgjk  +  gaki  +  wyk 

where  ‘u’  represents  an  overall  mean  effect;  the  variables  with  a  single  subscript  refer  to 
contributions  from  the  principal  factors  (aerial  surveillance,  breaching  system,  and  ground 
surveillance);  and  the  variables  with  more  than  one  subscript  are  cross  terms  or 
interactions  among  the  principal  factors  (aby,  bgjk,  and  ga^  represent  interactions  between 
aerial  surveillance-breacher,  ground  surveillance-breacher,  and  aerial-ground  surveillance 
interactions  respectively;  wyk  represents  interaction  among  all  three  factors).  All  indices 
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take  on  the  value  zero  or  one  (with  zero  connoting  the  “low  level”  of  a  factor),  and  all 
variables  are  subjected  to  the  constraint  that  any  sum  over  the  range  of  any  subscript  is 
zero,  e.g.  aboj.+  abij  =  0. 

By  fitting  the  above  model  to  each  of  the  three  measures  of  interest,  the  sets 
of  coefficients  in  Table  1  are  derived.  For  example,  the  expected  number  of  losses  to 
direct  and  indirect  fire  with  AVLM,  GSTAMIDS,  but  no  ASTAMIDS  is: 

2.156  +0.281  -0.094  +0.469  -0.219  -0.0313  +  0.344  -0.156  =  2.75, 

which  corresponds  to  the  fifth  vertical  bar  in  Figure  1,  above.  Note,  gi  =  0.469  appears 
in  the  expression  instead  of  go  =  -  0.469  because  GSTAMIDS  was  used  in  this  group  of 
trials. 


Table  1.  Regression  Coefficients  for  Main  Effects  and  Interactions 


Regression  Constant  or  Coefficient 

Blue  losses 

u 

ao 

bo 

9o 

a  boo 

bgoo 

gaoo 

Wood 

To  Mines 

1.53 

0.719 

0.344 

0.281 

0.656 

0.094 

0.219 

-0.094 

To  DF+IF 

2.156 

0.281 

-0.094 

-0.469 

-0.219 

0.0313 

-0.344 

0.156 

To  All 

Agents 

3.69 

1.0 

0.25 

-0.187 

0.438 

0.125 

-0.125 

0.0625 

The  ANOVA  tables  developed  in  the  body  of  this  report  indicate  that  ai  (aerial 
surveillance)  and  aby  (interaction  cross  term  between  aerial  surveillance  and  breaching 
system)  are  significant  with  respect  to  losses  to  mines,  whereas  only  ai  is  significant  with 
respect  to  total  losses.  The  “uneven”  behavior  mentioned  earlier  in  the  discussion  of 
Figure  1  and  the  significance  of  aby  in  the  linear  model  are  closely  related.  The 
suggestion  is  that  ASTAMIDS  data  caused  a  major  improvement  in  runs  where  AVLM 
was  available,  but  had  a  lesser  effect  when  ESMB  was  available.  This  may  be  due  to  the 
predilection  of  the  commander  to  circumvent  minefields  in  runs  with  AVLM  and 
ASTAMIDS  intelligence,  which  seems  not  to  be  present  when  ESMB  is  available. 
Indeed,  with  AVLM  and  ASTAMIDS,  Blue  breached  minefields  only  twice  in  eight  runs; 
without  ASTAMIDS  Blue  attempted  five  breaching  operations  in  the  same  number  of 
opportunities.  In  contrast.  Blue  launched  ESMB  in  five  runs  with  ASTAMIDS  and  in 
five  runs  without. 
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None  of  the  regression  terms  corresponding  to  principal  factors  is  significant  with 
respect  to  losses  to  direct  and  indirect  fire  unless  the  variance  due  to  different  areas  on 
which  the  replications  took  place  is  taken  into  account.  When  this  is  done,  then  gk, 
ground  surveillance,  becomes  significant  in  the  sense  that  GSTAMIDS  is  associated  with 
increased  losses  to  direct  and  indirect  fire  (see  discussion  below). 

By  examining  the  regression  coefficients  in  Table  1,  or  by  referring  to  Appendix 
A  for  the  data  supporting  Figure  1,  one  determines  that  the  average  number  of  Blue 
vehicles  lost  was  3.69  per  trial.  When  ASTAMIDS  intelligence  was  available,  losses 
dropped  to  2.69  per  trial.  Without  ASTAMIDS,  Blue  lost  4.69  vehicles  on  average. 
Losses  to  mines  as  well  as  losses  to  direct  and  indirect  fire  were  reduced  with 
ASTAMIDS.  Losses  to  mines  decreased  from  2.25  to  0.81,  while  losses  to  direct  and 
indirect  fire  dropped  from  2.44  to  1.88  per  trial.  While  it  would  be  reasonable  to  apply 
t-tests  to  determine  the  significance  of  the  difference  in  means  (for  example  the  difference 
in  mean  losses  for  the  trials  with  and  without  ASTAMIDS  is  significant  well  below  the 
10%  level),  the  analytic  route  followed  here  will  center  on  the  linear  model,  instead. 

The  use  of  other  technologies  (other  than  ASTAMINDS)  did  not  always  result  in 
fewer  Blue  losses.  For  example,  with  GSTAMIDS  Blue  lost  3.88  vehicles  per  trial,  but 
only  3.5  without.  While  Blue  losses  to  mines  were  reduced  with  GSTAMIDS  (1.25 
compared  to  1.8),  losses  to  direct  and  indirect  fire  increased  (2.63  with,  and  1.69 
without).  Similarly,  while  Blue  losses  overall  were  less  with  ESMB  than  with  AVLM 
(3.94  and  3.44,  respectively),  losses  to  direct  and  indirect  fire  were  slightly  higher  with 
the  newer  technology  (2.25  compared  to  2.06).  Losses  to  mines  were  considerably  less 
with  ESMB  (1.19)  than  with  AVLM  (1.88).  Table  2  summarizes  the  mean  number  of 
kills  of  Blue  vehicles  by  OPFOR  agents.  The  first  row  shows  the  technology  available  in 
the  16  trials  over  which  the  kills  were  averaged. 


Table  2.  Mean  Blue  Losses  by  Agent  Versus  Countermine  System 


Blue 

Losses 

AST 

No  AST 

ESMB 

AVLM 

GST 

No  GST 

By  mines 

0.81 

2.25 

1.19 

1.88 

1.25 

1.81 

By  D.F.+  I.F 

1.88 

2.44 

2.25 

2.06 

2.63 

1.69 

Total 

2.69 

4.69 

3.44 

3.94 

3.88 

3.5 
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The  impact  of  tactics,  that  is  minefield  avoidance  or  minefield  breaching,  was 
analyzed  as  a  covariable  to  the  basic  linear  model  (that  is,  it  was  included  in  a  second  set 
of  computations  as  a  regression  variable  without  interaction  with  any  of  the  terms 
associated  with  the  principal  factors).  Mentioned  earlier  as  a  possible  explanation  for 
interaction  between  the  breaching  systems  and  aerial  surveillance,  breaching  was 
attempted  or  initiated  in  19  of  the  32  trials.  Bypassing  was  conducted  in  the  remaining 
13.  Nine  “bypasses”  were  conducted  with  ASTAMIDS,  and  four  were  conducted 
without.  Similarly,  four  “bypasses”  were  conducted  when  ESMB  was  available  and  nine 
were  conducted  with  AVLM.  Unsurprisingly,  aerial  surveillance  data  and  type  of 
breaching  system  were  the  most  important  factors  determining  tactics. 

Blue  losses  in  trials  in  which  breaching  was  attempted  averaged  4.37  vehicles, 
compared  to  2.7  vehicles  in  trials  in  which  minefields  were  bypassed.  Losses  to  direct 
and  indirect  fire  was  essentially  the  same  for  both  (about  2.15).  Unsurprisingly,  losses  to 
mines  were  much  less  in  those  trials  in  which  bypassing  occurred  (0.54  compared  to  2.2). 
The  fact  that  any  occurred  at  all  was  apparently  due  to  poor  information  about  minefield 
locations,  the  inability  of  vehicles  to  navigate  gaps  between  fields,  or  simply  the  inability 
to  avoid  fields  due  to  their  layout  and  the  terrain.  Regarding  the  last  possibility,  all  losses 
to  mines  during  attempted  bypass  operations  occurred  in  Area  #1,  a  narrow  corridor  that 
was  effectively  blocked  by  mines. 

Finally,  learning  was  analyzed  through  stepwise  regression  and  did  not  appear  to 
have  an  impact,  even  when  the  effects  of  terrain  and  tactics  were  included.  On  the  other 
hand,  geographic  areas  were  significant.  In  particular.  Area  #1  stood  out  from  the  other 
subregions  as  more  costly  in  terms  of  both  losses  to  mines  and  to  direct  and  indirect  fire. 
The  explanation  of  why  this  was  the  case  is  unclear.  One  plausible  explanation  is  the  fact 
that,  since  replications  were  run  in  order  by  area.  Area  #1  was  the  first  region  on  which 
each  new  mix  of  equipment  was  used  (also.  Area  #1  was  topographically  more 
restrictive).  There  was  only  one  exception  to  this  procedure  in  the  test.  So,  perhaps 
learning  did  take  place  on  a  “local  level”:  with  respect  to  the  four  replications  of  each  mix 
of  countermine  equipment. 

There  are  three  points  that  stand  out  from  the  analyses  of  variance  of  CME  Hata- 
ASTAMIDS  intelligence  data  is  a  significant  factor  in  reducing  losses  due  to  mines  and 
all  agents  combined.  Secondly,  the  use  of  GSTAMIDS  is  costly  with  respect  to  losses 
from  direct  and  indirect  fire,  especially  in  breaching  operations.  The  third  and  final  point 
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is  that  there  appears  to  be  a  strong  interaction — with  respect  to  Blue  losses  to  mines — 
between  aerial  surveillance  and  the  explosive  breaching  system. 

Taking  these  points  one  at  a  time,  the  benefit  to  Blue  ascribed  to  ASTAMIDS  data 
is  due,  at  least  in  part,  to  the  fact  that  bypassing  (as  opposed  to  breaching)  minefields  was 
employed  as  a  tactic  at  a  relatively  high  frequency  when  ASTAMIDS  data  were  available. 
Breaching  was  costly:  overall  Blue  losses  were  fewer  when  minefields  were  avoided. 

Regarding  the  second  point,  it  is  difficult  to  say  precisely  why  Blue  losses  to 
direct  and  indirect  fire  increased  when  GSTAMIDS  was  deployed.  However,  it  is  very 
likely  that  Blue  vehicles  were  more  exposed  while  GSTAMIDS  was  used  to  find 
minefield  boundaries  and  gaps  between  fields.  Interaction  between  aerial  surveillance 
and  the  explosive  breaching  system  (the  third  point)  with  respect  to  mines  was  due  to  the 
fact  that  bypassing  was  the  preferred  tactic  when  AVLM  and  ASTAMIDS  data  were 
available.  The  possibility  of  a  tendency  (perhaps  ascribable  to  the  desire  to  exercise  a 
new  system)  to  breach  when  ESMB  was  on  hand  should  not  be  discounted.  When  tactics 
are  taken  into  consideration,  ESMB  reduces  losses  to  mines,  however. 

In  terms  of  the  linear  model,  only  aerial  surveillance  is  significant  with  respect  to 
losses  to  all  agents,  while  aerial  surveillance  and  the  interaction  between  aerial 
surveillance  and  breaching  system  are  significant  with  respect  to  losses  to  mines.  When 
the  variance  due  to  replications  (scenario  areas)  is  taken  into  account,  ground  surveillance 
becomes  significant  with  respect  to  losses  to  direct  and  indirect  fire.  Finally,  when  tactics 
and  scenario  areas  are  included  as  covariables  to  the  basic  model,  the  resulting  significant 
factors  with  respect  to  losses  to  mines  are:  tactics,  scenarios,  breaching  systems,  and  the 
interaction  between  aerial  surveillance  and  breaching  system.  With  the  exception  of  the 
interaction,  the  same  factors  are  significant  with  respect  to  losses  to  all  agents.  Finally, 
ground  surveillance  and  scenario  area  are  significant  with  respect  to  direct  and  indirect 
fire  losses. 

The  issues  raised  in  this  analysis  of  CME  should  be  investigated  in  greater  detail. 
An  appropriate  research  mechanism  might  be  a  constructive  model  in  which  large 
numbers  of  trials  can  be  generated.  It  is  highly  recommended  that  future  test  designs 
include  tactics  (breaching  -  bypassing)  as  a  principal  factor. 

Finally,  the  conclusions  reached  in  this  analysis  largely  agree  with  those  in  IDA  Paper  P- 
3300.  A  principle  difference  is  the  fact  that  the  current  analysis  identifies  GSTAMIDS 
as  a  significant  factor  in  Blue  losses  to  direct  and  indirect  fire.  Another  is  the 
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identification  of  the  impact  of  Scenario  Area  1.  Both  studies,  however,  suggest 
ASTAMHDS  data  was  a  significant  factor  in  reducing  Blue  losses  to  mines  and  all  agents 
combined. 
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PART  2 
ANALYSES 
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I.  CONDUCT  OF  THE  TEST 


A.  PRELIMINARY  PHASE 

The  CME  was  conducted  in  two  phases:  a  1-week  preliminary  phase  in  May  and 
June  1996,  in  which  aerial  surveillance  data  were  collected  and  a  4- week  combat  phase  in 
July  1996.  The  preliminary  phase  was  a  non-combat  activity  in  which  ASTAMIDS 
missions  were  flown  (simulated)  over  the  regions  in  which  the  July  test  was  to  be 
conducted.  Minefields  in  the  preliminary  phase  were  emplaced  in  exactly  the  same 
location  as  those  in  the  principal  phase  and  detection  data  collected  by  ASTAMK)S  were 
provided  to  the  Blue  force  in  the  appropriate  trials.  The  purpose  of  this  preliminary  phase 
was  to  perform  intelligence  gathering  missions  “off-line”  in  order  to  conserve  time  during 
the  principal  phase  of  CME  as  well  as  replicate  the  execution  of  surveillance  activities  in 
advance  of  combat  missions. 

B.  EQUIPMENT 

The  CME  combat  phase  was  a  free  play  exercise  in  which  a  Blue  armored 
company,  supported  by  an  engineering  platoon  and  artillery,  engaged  a  small  OPFOR  unit 
supported  by  minefields  and  artillery.  The  Blue  force  consisted  of  one  MlAl  armored 
company,  an  M113A2  engineering  platoon,  and  an  additional  MlAl  platform  for  the 
breaching  system  or  ground  surveillance  system  or  both.  Five  of  the  Blue  vehicle 
simulators  had  human  operators:  the  armor  company  commander’s  MlAl  and  each  of 
the  Blue  platoon  leader’s  (three  tank  and  one  engineer)  vehicles.  The  unmanned  portions 
of  the  engineering  platoon  and  each  tank  platoon  were  simulated  as  Modular  Semi- 
Automated  Force  (ModSAF)  entities.  Each  was  controlled  by  an  individual  operator  and 
a  separate  workstation.  OPFOR  vehicles,  two  BMPs  and  one  T80  were  likewise 
controlled  by  one  operator  and  a  single  workstation. 

The  equipment  available  to  each  side  is  listed  in  Table  3,  below.  Five  of  the 
unmanned  MlAls  were  equipped  with  either  rollers  or  plows.  In  addition  to  these 
mechanical  mine  clearing  systems.  Blue  also  had  one  explosive  mine  clearing  system: 
either  the  trailer  mounted  Explosive  Standoff  Mine  Breacher  (ESMB),  or  the  tank- 
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mounted  Armor  Vehicle  Launched  Mine  Clearing  Line  Charge  (AVLM).  Two 
surveillance  systems  were  available  to  Blue  (depending  on  the  trial  as  described  in  the 
trial  matrix  below).  These  were  the  Airborne  Standoff  Minefield  Detection  System 
(ASTAMIDS)  and  the  Ground  Standoff  Minefield  Detection  System  (GSTAMIDS). 


Table  3.  Blue  and  OPFOR  Equipment 


Equipment 

Blue 

OPFOR 

Armor 

13M1A1S 

(3withpiows,  2  with 
rollers) 

1  T80 

2BMP2 

Engineer  vehicles 

4  M113A2 

Mines 

4  to  6  minefields 

140  AT 

80  AP 

(each  field) 

CM  equipment 

AVLM 

ESMB 

GSTAMIDS 

ASTAMIDS 

Artillery 

M107  unit 

Bomb  button 

Blue  countermine  systems  other  than  rollers,  plows,  and  AVLM  were  simulated  as 
Dial-a-Tank  entities,  developed  by  MdK  Technologies.  Rollers,  plows,  and  AVLM  were 
simulated  in  ModSAF.  Minefields  were  created  and  controlled  by  the  Comprehensive 
Mine  Simulator  (CMS). 

In  addition  to  their  crews,  each  manned  vehicle  contained  a  research  assistant  who 
was  assigned  to  take  notes  throughout  the  trial.  Research  assistants  were  also  assigned  to 
each  of  the  ModSAF  operators.  The  battalion  commander,  who  served  as  coordinator  and 
overseer  of  the  CME,  also  recorded  notes  throughout  the  trial.  In  addition  to  these  notes, 
data  were  recorded  on  video  tape  and,  most  importantly  for  this  paper,  by  data  logger 
software. 
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C.  PREPARATION 

Trials  were  conducted  twice  a  day  over  a  4-week  period.  Pre-briefs  were 
conducted  prior  to  each  run.  These  meetings  consisted  of  a  lecture  by  the  company 
commander  to  the  Ml A1  and  Ml  13  crews  in  which  he  described  the  objective,  tactics, 
and  intelligence  regarding  and  minefield  locations  and  OPFOR  armor.  These  sessions 
took  place  in  the  presence  of  the  battalion  commander  and  represented  the  company 
commander’s  synthesis  of  the  battalion  commander’s  fragmentary  orders  (FRAGOs). 

The  trials  began  immediately  after  the  pre-briefs  and  were  conducted  under  the 
direction  of  the  battalion  commander.  It  was  his  responsibility  to  enforce  ground  rules 
and  determine  stopping  conditions.  The  battalion  commander  also  conducted  after-action 
reviews  in  which  soldier  “feedback”  was  solicited  and  recorded. 

D.  TEST  MATRIX 

The  experiment  consisted  of  32  simulation  runs  in  which  8  variations  (all  possible 
combinations  of  three  factors)  of  countermine  equipment  were  available  to  Blue.  A  given 
mix  of  equipment  was  used  in  four  successive  runs,  each  of  which  took  place  in  a 
different  geographical  area  of  the  Ft.  Knox  terrain  database.  The  countermine  equipment 
consisted  of  one  of  two  explosive  breaching  devices,  AVLM  or  ESMB,  a  ground-based 
mine  detection  system,  GSTAMIDS,  and  an  aerial  surveillance  system,  ASTAMIDS. 
The  equipment  mixes  were: 

1 .  AVLM  alone 

2.  AVLM  and  ASTAMIDS 

3.  ESMB  alone 

4.  ESMB  and  ASTAMIDS 

5.  GSTAMIDS  and  AVLM 

6.  GSTAMIDS,  AVLM  and  ASTAMIDS 

7.  ESMB-GSTAMIDS  combined 

8.  ESMB-GSTAMIDS  with  ASTAMIDS. 

Table  4  is  a  test  matrix  that  indicates  the  order  in  which  the  tests  were  conducted 
and  the  various  mixes  of  equipment  that  were  available  to  Blue  in  each  run.  With  the 
exception  of  the  first,  each  row  corresponds  to  a  different  week  of  the  exercise,  while  the 
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adjacent  columns  correspond  (roughly)  to  mornings  and  afternoons  during  the  test  period. 
In  a  typical  week,  eight  runs  would  take  place  with  one  day  reserved  for  make-ups  and 
administrative  activities. 


Table  4.  Test  Matrix 


AST 

AST 

AST 

AST 

area  1 

area  2 

area  3 

area  4 

area  1 

area  2 

area  3 

area  4 

AVLM  (base  case) 

29 

30 

31 

32 

1 

2 

3 

4 

ESMB 

5 

6 

7 

8 

12 

9 

10 

11 

AVLM-GST 

13 

14 

15 

16 

17 

18 

19 

20 

ESMB-GST 

21 

22 

23 

24 

25 

26 

27 

28 

In  the  parlance  of  the  test  conductors,  each  row  was  referred  to  as  a  “vignette.” 
These  were  intended  to  correspond  to  successive  weeks  of  the  test;  however,  some 
repetition  of  base  case  runs  took  place  during  fourth  week  (hence  the  irregular 
numbering).  Thus  the  runs  in  the  first  row  in  which  AVLM  was  the  breaching  device 
were  referred  to  collectively  as  “vignette  1.”  The  runs  in  which  ESMB  and  GSTAMIDS 
were  used  together  were  referred  to  as  “vignette  4.” 

E.  FACTORIAL  DESIGN 

The  test  matrix  falls  into  what  is  classically  known  as  a  “2^  factorial”  design. 
Three  factors,  the  breaching  device,  ground  surveillance,  and  aerial  surveillance,  each 
appear  in  two  “levels.”  In  this  scheme  AVLM  and  ESMB  can  be  thought  of  as  the  low 
and  high  levels  of  explosive  breachers,  respectively.  The  low  level  of  ground 
surveillance  is  associated  with  not  having  GSTAMIDS,  while  the  high  level  is  the 
condition  of  having  GSTAMIDS  available.  Similarly,  the  low  and  high  levels  of  aerial 
surveillance  correspond  to  not  having  or  having  ASTAMIDS  data,  respectively.  Finally, 
the  four  geographic  areas  in  which  each  mn  is  repeated  with  identical  equipment  mixes 
corresponds  to  taking  repeated  measurements  (4)  for  each  combination  of  the  “principal 
factors.” 

Pictorially,  this  test  design  can  be  represented  by  a  cube.  Each  vertex  represents 
or  corresponds  to  a  specific  mix  of  countermine  equipment  (again,  principal  factors). 
Thus,  in  Figure  2,  the  top  plane  corresponds  to  the  high  level  of  aerial  surveillance — 
those  runs  in  which  ASTAMIDS  was  used — while  the  back  plane  represents  those  in 
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which  ESMB  was  available.  Similarly,  the  rightmost  plane  corresponds  to  the  high  level 
of  ground  surveillance. 


ASTAMIDS 


Figure  2.  Test  Design 

The  utility  of  this  representation  is  the  opportunity  to  analyze  outcomes 
(measures)  both  in  terms  of  the  contribution  of  each  of  the  principal  factors  and  in  terms 
of  the  interaction  of  these  factors  with  one  another.  This  is  particularly  simple  to  do 
because  the  test  design  is  balanced  (four  repeated  measures  take  place  at  each  “vertex”) 
and  can  easily  be  described  in  terms  of  a  standard  ANOVA  table  for  a  given  Measure  of 
Effectiveness  (MOE). 

From  a  mathematical  perspective,  the  expected  outcome  of  a  given  run  (or  the 
value  associated  with  a  given  vertex  on  the  cube)  is 

E(yijk„)  =  u  +  ai  +  bj  +  gk  +  aby  +  bgjk  +  gaki  +  wyk 

where  ‘u’  represents  an  overall  mean  effect,  the  variables  with  a  single  subsript  refer  to 
contributions  from  the  principal  factors  (aerial  surveillance,  breaching  system,  and  ground 
surveillance),  and  the  variables  with  more  than  one  subscript  are  cross  terms  or 
interactions  among  the  principal  factors.  The  difference  between  the  expected  outcome 


15 


UNCLASSIFIED 


UNCLASSIFIED 


and  the  actual  outcome,  yyim  ,  is  the  random  error,  which  is  assumed  to  be  normally 
distributed  with  zero  mean  and  unknown  variance. 

Since  there  are  only  two  levels  of  each  factor  in  our  model,  each  subscript  takes 
on  only  two  values  (zero  and  one,  say).  Further,  the  indexed  variables  are  constrained  by 
the  requirement  that  their  sum  over  the  range  of  a  given  index  is  zero  (essentially  to  avoid 
over-parameterization  and  allow  the  analyst  to  separate  principal  effects  in  the  presence 
of  interactions).  That  is, 

£  aj  =  X  bj  =  X  gk  =0 

For  multiply-indexed  variables,  such  as  aby,  the  constraint  is  Xi  aby  =  Xj  aby  =  0. 
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II.  ANALYSES 


A.  BLUE  LOSSES 

Throughout  most  of  this  paper,  the  MOE  of  interest  will  be  the  number  of  Blue 
vehicles  killed  in  a  trial  or  set  of  trials.  As  the  Opposing  Force  (OPFOR)  had  at  its 
disposal  direct  fire  weapons  (T80s  and  BMPs),  indirect  fire,  and  mines,  it  will  sometimes 
be  of  interest  to  group  Blue  losses  by  agent,  as  in  Figure  3.  In  most  cases,  however,  kills 
by  direct  and  indirect  fire  will  be  grouped  together  and  distinguished  from  kills  by  mines. 
Also,  the  term  “kills”  will  refer  to  all  levels  of  Blue  vehicular  losses.  Thus,  the  number  of 
“kills”  in  a  single  trial  will  refer  to  the  sum  of  mobility,  firepower,  combined  mobility  and 
firepower,  and  catastrophic  kills — with  at  most  only  one  level  of  kill  per  vehicle  entering 
the  tally. 


Total  Kills  of  Blue  Vehicles  by  Agent 


BASE  ESMB  GST  COMBO 


CM  Configuration 


Figure  3.  Blue  Losses  by  Agent 

Through  most  of  this  analysis,  however,  we  will  be  interested  in  the  average 
number  of  kills  occurring  in  the  four  runs  in  which  countermine  equipment  was  held 
fixed.  In  that  vein,  Figure  3  shows  average  Blue  losses  by  agent  for  all  the  mixes  of 
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countermine  equipment.  Reading  left  to  right,  the  graph  corresponds  to  countermine 
conditions  (CM)  present  in  vignettes— or  weeks — one  through  four. 

Reeall  from  the  Executive  Summary,  where  this  figure  was  first  presented,  that 
the  average  number  of  Blue  vehicles  lost  was  3.69  per  trial.  When  ASTAMIDS 
intelligence  was  available,  losses  dropped  to  2.69  per  trial.  Without  ASTAMIDS,  Blue 
lost  4.69  vehicles  on  average.  Losses  to  mines  as  well  as  losses  to  direct  and  indirect  fire 
were  reduced  with  ASTAMIDS.  Losses  to  mines  decreased  from  2.25  to  0.81,  while 
losses  to  direet  and  indirect  fire  dropped  from  2.44  to  1.88  per  trial.  (Recall  that  each  bar 
in  Figure  3  represents  the  average  of  four  mns.) 

Table  5,  below,  groups  the  data  presented  in  Figure  3  so  that  the  effects  of  the 
presence  versus  the  absence  of  CM  systems  can  be  compared,  system  by  system.  For 
example,  one  sees  that  the  average  Blue  loss  without  ASTAMIDS  is  about  4.7  vehicles, 
while  with  ASTAMIDS,  losses  are  reduced  to  about  2.7  vehieles.  Similarly,  there  is  a 
reduction  in  loses  to  mines  from  2.25  to  0.81  vehicles  when  ASTAMIDS  is  deployed. 
While  two-tailed  t-tests  show  both  of  these  to  be  significant  reductions,  it  will  be  the 
focus  of  this  paper  to  analyze  outcomes  in  terms  of  ANOVA  tables  and  not  comment  too 
heavily  on  the  significance  of  the  difference  in  means. 


Table  5.  Mean  Blue  Losses  by  Agent  Versus  Countermine  System 


Blue 

Losses 

AST 

No  AST 

AVLM 

ESMB 

GST 

No  GST 

By  mines 

0.81 

2.25 

1.88 

1.19 

1.25 

1.81 

By  D.F.+  I.F 

1.88 

2.44 

2.06 

2.25 

2.63 

1.69 

Total 

2.69 

4.69 

3.94 

3.44 

3.88 

3.5 

Interestingly,  there  is  an  increase  in  total  losses  from  3.5  to  3.88  vehicles  when 
GSTAMIDS  is  deployed.  Although  there  is  a  decrease  in  losses  to  mines,  losses  to  direct 
and  indirect  fire  more  than  make  up  the  difference.  The  latter  fluctuation,  from  1.69  to 
2.63  vehicles,  is  likely  due  to  the  amount  of  time  Blue  forces  are  stationary  and  exposed 
to  indirect  fire  while  minefields  are  probed  with  GSTAMIDS.  While  neither  the  increase 
in  losses  to  all  agents  nor  the  decrease  in  losses  to  mines  is  significant,  the  change  in 
[direet  plus  indirect]  fire  losses  is.  A  two-tailed  paired  comparison  shows  this  difference 
in  means  is  significant  below  the  10%  level. 
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B.  OPFOR  LOSSES  AND  LOSS  EXCHANGE  RATIO 

OPFOR  deployed  only  three  vehicles  in  each  run:  one  T80  and  two  BMP2s. 
OPFOR  artillery  was  controlled  by  the  ModSAF  “Bomb  Button,”  a  mechanism  that  was 
invulnerable  to  Blue  countermeasures  and  controlled  by  the  ModSAF  operator.  The 
variability  of  OPFOR  losses  was  very  small.  OPFOR  lost  two  vehicles  in  12  cases,  all  3 
vehicles  in  18  cases,  and  1  vehicle  in  each  of  the  remaining  2.  The  mean  was  2.5  with 
standard  error  of  the  mean  approximately  equal  to  0.1 1.  Thus  the  distribution  of  OPFOR 
losses  is  not  particularly  interesting  and  the  graphs  of  Blue  losses  essentially  relates  all 
the  information  regarding  the  exchanges  between  OPFOR  and  Blue. 

Having  stated  that  OPFOR  losses  do  not  provide  very  interesting  statistics,  a 
graph  (Figure  4)  of  the  loss  exchange  ratios  is  included  (Blue  losses/OPFOR  losses)  for 
completeness.  As  expected,  it  follows  the  general  pattern  of  Blue  losses  as  shown  in  the 
earlier  graph.  Figure  3. 


Exchange  Ratios  (Blue/OPFOR) 


2.50 
2.00 

1.50 
1.00 
0.50 
0.00 


AVLM  ESMB  AVLM-GST  ESMB-GST 
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B  no  AST 
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Figure  4.  Loss  Exchange  Ratio 


C.  INTERACTIONS 

In  a  factorial  design,  the  working  hypothesis  is  that  the  expected  outcome  in  a 
given  trial  is  a  sum  of  components,  one  from  each  main  factor  plus  a  collection  of  “cross” 
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terms  that  represent  interaction  among  the  factors.  The  observed  outcome  is  this 
expected  value  plus  a  random  error.  This  section  is  a  detailed  discussion  of  the 
interactions  among  the  countermine  systems  that  are  the  major  factors  in  this  experiment. 

Since  the  various  types  of  countermine  equipment  used  in  these  experiments 
would  typically  be  used  in  concert  with  one  another,  (e.g.,  a  breacher  would  be  fired 
where  a  detection  system  indicated  a  minefield  exists),  some  evidence  of  interaction  is 
expected.  A  measure  of  the  interaction  between  two  factors,  say  ground  surveillance  and 
breaching  systems,  is  obtained  by  comparing  the  average  outcomes  at  the  high  and  low 
levels  of  each  factor.  More  precisely,  interaction  is  measured  by  comparing  the 
differences  in  outcomes  between  the  high  and  low  levels  of  one  factor  at  each  of  the  two 
levels  of  the  second  factor.  Heuristically,  this  is  equivalent  to,  first,  collapsing  the  cube  in 
Figure  2  along  its  vertical  axis  and  computing  average  outcomes  for  each  of  the  vertices 
of  the  resulting  square.  Then,  select  opposite  edges  of  the  square  and  compare  the 
difference  in  outcomes  between  vertices  on  one  edge  with  the  difference  on  the  other. 
The  extent  to  which  these  two  deltas  differ  determines  the  degree  to  which  the  factors 
interact. 

For  example.  Table  6  below  shows  average  Blue  losses  with  and  without 
GSTAMDDS  for  the  cases  in  which  AVLM  was  available  and  those  in  which  ESMB  was 
available.  The  bottom  row  (denoted  “Delta”)  shows  the  difference  between  losses  with 
and  without  GSTAMIDS  for  runs  with  AVLM  and  runs  with  ESMB. 


Table  6.  Interaction  Between  Ground  Surveillance  and  Breaching  Systems 


Blue 

Losses 

By  All  Agents 

By  DF+IF 

By  Mines 

AVLM 

ESMB 

AVLM 

ESMB 

AVLM 

ESMB 

No  GST 

3.875 

3.125 

1.625 

1.75 

2.25 

1.375 

GST 

4 

3.75 

2.5 

2.75 

1.5 

1 

Delta 

-0.125 

-0.625 

-0.875 

-1 

0.75 

0.375 

For  a  given  category  of  agent,  say  mines,  the  deltas  are  0.75  and  0.375.  These  two 
values  are  too  close  to  suggest  any  significant  interaction  between  ground  surveillance 
and  breaching  system.  A  more  precise  indication  of  significance  will  appear  in  the 
ANOVA  tables  of  a  later  section.  The  following  graph.  Figure  5,  gives  a  pictorial 
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representation  of  the  data  in  Table  6.  The  distance  between  the  ends  of  the  line  segments 
are  the  deltas.  The  interaction  is  the  extent  to  which  the  lines  diverge  from  parallel. 


Figure  5.  Graphical  Representation  of  Interaction  Between 
Ground  Surveillance  and  Breaching  Systems 

Interactions  between  aerial  surveillance  and  breaching  systems  are  markedly 
different  from  the  previous  case,  as  can  be  seen  in  Table  7.  For  example,  when  AVLM  is 
available,  losses  to  mines  are  considerably  greater  without  ASTAMIDS  than  with 
ASTAMLDS.  But,  on  the  other  hand,  when  ESMB  is  available,  aerial  surveillance  does 
not  appear  to  affect  losses  to  mines.  Thus  deltas  differ,  indicating  an  interaction  between 
these  factors.  Some  level  of  interaction  exists  with  respect  to  the  other  measures,  losses 
to  all  agents  and  losses  to  direct  and  indirect  fire,  also.  However,  it  is  not  as  pronounced 
as  the  losses  to  mines. 


Table  7.  Interactions  Between  Aerial  Surveillance  and  Breaching  Systems 


Blue 

Losses 

By  All  Agents 

By  DF+IF 

By  Mines 

AVLM 

ESMB 

AVLM 

ESMB 

AVLM 

ESMB 

No  AST 

5.375 

4 

2.125 

3.25 

AST 

2.5 

2.875 

2 

0.5 

1.125 

Delta 

2.875 

1.125 

0.125 

1 

2.75 

0.125 
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A  striking  feature  in  Figure  6,  below,  is  the  huge  decrease  in  losses  to  mines  (and 
to  all  agents  combined)  realized  when  ASTAMIDS  is  used  in  conjunction  with  AVLM. 
This  bears  some  scrutiny  and  skepticism  for  the  following  reason.  Recall  that  there  are 
eight  runs  in  which  ASTAMIDS  and  AVLM  were  both  available  to  Blue.  In  only  two  of 
these  did  Blue  actually  deploy  AVLM:  in  the  remaining  six.  Blue  circumvented 
minefields  detected  by  ASTAMIDS.  Without  ASTAMIDS,  Blue  initiated  breaching 
operations  with  AVLM  in  five  out  of  eight  runs.  With  ESMB,  Blue  initiated  breaching 
operations  in  five  out  of  eight  trials  with  ASTAMIDS  and  in  five  out  of  eight  trials 
without  ASTAMIDS  (Blue  also  conducted  breaching  operations  with  rollers  and  plows  in 
two  additional  runs:  in  at  least  one  of  which  ESMB  was  destroyed  by  mines.)  While  a 
comparison  of  breaching  and  bypassing  minefields  will  be  the  subject  of  a  later  analysis 
section,  it  is  worth  saying  at  this  point  that  losses  to  mines  were,  on  average,  2.2  vehicles 
in  runs  where  breaching  operations  were  attempted,  and  0.54  where  they  were  not. 
Losses  to  direct  and  indirect  fire  were  about  the  same  (2.15)  in  each  case. 


Figure  6.  Graphical  Representation  of  Interactions  Between 
Aerial  Surveillance  and  Breaching  Systems 
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Aerial  and  ground  surveillance  make  up  the  final  system  pair  to  be  considered.  As 
shown  in  Table  8,  the  most  interesting  interaction  appears  in  terms  of  losses  to  direct  and 
indirect  fire.  Without  GSTAMIDS,  there  is  little  difference  between  having  or  not  having 
aerial  surveillance.  With  GSTAMIDS,  there  appears  to  be,  relatively,  a  large  difference. 


Table  8.  Interactions  Between  Aerial  and  Ground  Surveillance  Systems 


By  All  Agents 

By  DF+IF 

By  Mines 

Blue 

Losses 

No  GST 

GST 

No  GST 

GST 

No  GST 

GST 

No  AST 

4.375 

5 

3.25 

2.75 

AST 

2.625 

2.75 

IBHI 

2 

0.875 

^■1 

Delta 

1.75 

2.25 

-0.125 

1.25 

1.875 

The  graph  in  Figure  7  demonstrates  the  interaction  fairly  vividly.  Since  the  deltas 
for  direct  and  indirect  fire  have  different  signs,  the  graphs  of  average  losses  intersect. 
Generally  speaking,  this  is  the  signature  of  interaction  between  systems. 


Average  Loss  by  Agent  versus  Surveillance  System 


Figure  7.  Graphical  Representation  of  Interaction  between 
Aerial  and  Ground  Surveillance  Systems 
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The  reason  for  the  observed  effect  is  less  clear  than  in  the  previous  case  regarding 
AVLM  and  ASTAMIDS.  Here,  ASTAMIDS  has  some  payoff  for  Blue  in  general,  while 
GSTAMIDS  appears  to  be  associated  with  an  increase  in  losses  to  direct  and  indirect  fire 
when  used  without  aerial  surveillance.  This  observation  may  be  related  to  the  amount  of 
time  the  Blue  force  spent  in  an  exposed  position  while  GSTAMIDS  searched  for  the 
leading  edge  of  the  minefield  in  preparation  for  launching  an  explosive  breacher  or, 
possibly,  for  gaps  between  the  fields.  This  issue  bears  further  investigation  and  may  be 
an  appropriate  focus  for  a  constructive  model. 

D.  ANALYSIS  OF  VARIANCE 

This  section  is  an  attempt  to  put  some  of  the  tables  and  graphs  in  the  previous 
section  on  a  statistical  footing.  The  primary  mechanism  to  be  used  is  classical  analysis  of 
variance  (ANOVA)  because  the  exercise  is  based  on  a  balanced  factorial  design  and  lends 
itself  easily  to  this  technique.  As  used  here,  ANOVA  decomposes  the  sum  of  squared 
deviations  of  Blue  losses  over  the  entire  experiment  into  components  associated  with  the 
breacher,  the  two  detection  systems,  and  their  interactions.  This  decomposition  enables 
the  analyst  to  isolate  the  most  influential  effects  by  performing  one  test.  An  F-statistic 
relates  each  component  (of  the  sum  of  squares)  to  the  probability  that  deviations  as  large 
as  those  observed  would  occur  by  chance.  A  large  value  of  F  (depending  on  the  degrees 
of  freedom  involved)  indicates  a  significant  outcome.  For  example,  in  the  tables  of  F- 
statistics  given  below,  the  large  values  accompanying  aerial  surveillance  are  indicative  of 
the  impact  this  system  has  on  Blue  losses. 

Complete  ANOVA  tables  for  all  factors  and  interactions  is  deferred  until 
Appendix  B.  Only  the  F-statistics  and  the  corresponding  principal  factors  and 
interactions  are  presented  here.  In  keeping  with  the  previous  format,  tables  are  shown  for 
total  Blue  losses,  losses  due  to  direct  and  indirect  fire  combined,  and  losses  to  mines. 
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Table  9.  F-Statistics  for  Main  Effects  and  Interactions 


Source  of  Variation 

F-  Statistics 

Losses  To  Ail  Agents 

To  DF+IF 

To  Mines 

Breaching  System  (BR) 

0.2 

0.084 

1.12 

Ground  Surveillance 

0.11 

2.1^ 

0.75 

Aerial  Surveillance 

3.12““ 

0.76 

4.88' 

Interact  (BR  -  Ground) 

0.05 

0.0093 

0.08 

Interact  (BR  and  Aerial) 

0.6 

0.46 

4.07* 

Interact  (Aerial-Ground) 

0.05 

1.13 

0.45 

Interact  (BR-Ground-Aerial) 

0.12 

0.23 

0.83 

R^ 

0.147 

0.166 

0.323 

Increases  to  3.38,  which  is  siQnificant  at  the  1 0%  level,  when  variance  due  to  replications  (area)  is  taken 


into  account  (see  Section  F  and  Appendix  B). 

Significant  at  the  10%  level. 

®  Significant  at  the  5%  level. 

Table  9  indicates  that  the  factor  having  greatest  impact  on  losses  to  all  agents  and 
losses  to  mines  is  aerial  surveillance.  This  is  in  keeping  with  the  discussion  following 
Figure  3,  which  showed  that  mean  losses  with  and  without  ASTAMIDS  differed 
significantly.  As  indicated  in  the  earlier  discussion,  variations  due  to  other  systems  do 
not  have  as  great  an  impact.  Interactions  were  significant  between  aerial  surveillance  and 
breaching  systems  (with  respect  to  losses  to  mines  and  all  agents),  but  not  between  aerial 
and  ground  surveillance  as  anticipated  in  the  last  section — even  with  respect  to  losses  to 
direct  and  indirect  fire.  Again,  this  indicates  that  the  difference  in  outcomes  with  and 
without  ASTAMIDS  varies  so  greatly  from  AVLM  to  ESMB,  that  the  variation  is 
unlikely  to  be  ascribed  to  chance.  However,  the  corresponding  differences  for  aerial  and 
ground  surveillance,  even  for  losses  to  direct  and  indirect  fire,  are  not  large  enough  when 
compared  to  the  random  behavior  of  the  system  to  be  considered  significant. 

A  word  of  caution  regarding  the  impact  of  aerial  surveillance  is  appropriate. 
Since  there  is  significant  interaction  between  aerial  surveillance  and  breaching  systems, 
the  actual  effect  of  aerial  surveillance  is  not  estimable  without  the  somewhat  artificial 
constraint  that  indexed  terms  sum  to  zero  in  the  linear  model  (Searle,  Linear  Models  for 
Unbalanced  Data,  John  Wiley,  1987,  pp.  331-332).  The  difficulty  lies  in  the  fact  that  the 
mean  effects  of  the  two  levels  of  aerial  surveillance  cannot  be  separated  from  the 
influence  of  the  breachers. 

In  Section  n.  C,  above,  there  is  some  discussion  of  a  possible  link  between  the 
improved  outcomes  when  Blue  deployed  ASTAMIDS  and  AVLM,  and  the  tendency  to 
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conduct  bypassing  or  minefield  avoidance  operations  when  these  two  systems  were 
available.  Table  9  reaffirms  this  observed  effect  in  terms  of  the  F-statistic.  Because  of 
the  technical  evidence  of  this  interaction  and  its  possible  relation  to  decisions  to  conduct 
bypass  operations,  Night  Vision  Laboratory  requested  that  the  experiment  be  re-examined 
from  the  standpoint  of  a  factorial  design  in  which  the  tactic,  bypassing  or  breaching,  is 
examined  as  a  main  effect  with  two  levels.  A  variation  of  this  approach  is  the  subject  of  a 
later  section. 

Before  continuing  on  to  the  next  section,  however,  it  may  be  of  interest  to 
compute  the  regression  coefficients  of  the  model  proposed  in  Section  IE,  above.  These 
coefficients  belong  to  the  model  dictated  by  the  2^  factorial  design  and  correspond  to  the 
variables  (including  a  constant  term)  contained  in  Table  10. 


Table  10.  Regression  Coefficients  for  Main  Effects  and  interactions 


Value  of  Regression  Constant  or  Coefficient 

Losses  To  All 
Agents 

To  DF+IF 

To  Mines 

u  (constant) 

3.69 

2.156 

1.53 

ao  aerial  surveillance 

1.0 

0.281 

0.719 

bo  breaching  system 

0.25 

-0.094 

0.344 

go  ground  surveillance 

-0.187 

-0.469 

0.281 

aboo  Interact  (BR  and  Aerial) 

0.438 

-0.219 

0.656 

bgoo  Interact  (BR  -  Ground) 

0.125 

0.0313 

0.094 

gaoo  Interact  (Aerial-Ground) 

-0.125 

-0.344 

0.219 

wooo  Interact  (BR-Ground-Aerial) 

0.0625 

0.156 

-0.094 

The  zero  subscript  in  the  above  table  denotes  the  lower  level  of  the  system  (e.g., 
ao  means  ASTAMINDS  is  not  used).  The  remaining  coefficients,  such  as  ai,  are 
determined  from  the  constraints  (X  ai  =  0,  etc.)  introduced  with  the  linear  model  in 
Section  I.E,  Factorial  Design.  For  example,  ai  =  -1.0  when  the  measure  is  “loss  to  all 
agents”  and  -0.72  when  the  measure  is  “loss  to  mines.”  Taking  this  a  step  further,  the 
expected  loss  to  all  agents  under  the  full  linear  model  without  ASTAMIDS,  GSTAMIDS, 
or  ESMB  is 

E(yooon)  =  3.69  +1.0  +0.25  -0.187  +  0.438  +0.125  -0.125  +  0.0625  =  5.25; 
when  ASTAME)S  is  included,  the  expected  loss  becomes 

E(yioon)  =  3.69  -1.0  +  0.25  -0.187  -  0.438  +  0.125  +  0.125  -  0.0625  =  2.5. 
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Both  of  these  agree  with  the  average  observed  outcomes,  as  is  the  case  for  all  remaining 
cells,  also. 

E.  LEARNING 

A  natural  question  in  tests  of  this  sort  is  whether  or  not  learning  had  a  measurable 
impact.  In  the  first  half  of  the  trials.  Blue’s  losses  to  all  agents  were,  on  average,  3.6 
vehicles  per  trial.  In  the  second  half,  it  was  about  3.8.  Losses  to  mines  were  about  1.56 
and  1.5  vehicles  per  trial  in  the  first  and  second  halves,  respectively.  Finally,  losses  to 
direct  and  indirect  fire  were  2.04  vehicles  per  trial  in  the  first  half  and  2.3  in  the  second. 
These  data  seem  to  indicate  that  learning  was  not  a  factor.  However,  since  the  systems 
used  in  the  second  half  were  different  from  those  used  in  the  first,  an  analysis  that  takes 
the  different  systems  into  account  needed  to  be  performed.  Consequently,  a  stepwise 
regression  algorithm  was  applied  in  which  the  starting  variables  are  those  of  the  linear 
model  plus  the  sequence  in  which  the  test  trials  occurred  (see  Table  4).  The  algorithm 
(provided  in  SPSS  version  7.5),  which  only  enters  variables  with  less  than  a  10%  level  of 
significance,  yields  the  set  of  coefficients  and  F-statistics  appearing  below  in  Tables  1 1 
and  12. 


Table  11.  Stepwise  Regression  Constant  and  Coefficients 


Value  of  Regression  Constant  or  Coefficient 

Losses  To  All 
Agents 

ToDF+IF 

To  Mines 

u  (constant) 
ao  (aerial) 

aboo  (aerial-breaching  system  Interact) 

3.69 

2.156 

1.53 

IHHB! 

0.719 

0.656 

Table  12.  F-Statistics  Corresponding  to  Stepwise  Regression 


F-statistics 

Losses  To  All 
Agents 

To  DF+IF 

To  Mines 

ao  (aerial) 

aboo  (aerial-breaching  system  interact) 

5.35 

1  4.46 

As  the  “learning”  covariate  does  not  appear  in  the  tables,  the  order  of  the  trials 
was  apparently  not  a  significant  factor.  This  contention  is  also  supported  by  other 
regression  analyses  in  which  terrain,  that  is,  scenario  area,  and  tactics  are  taken  into 
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account.  On  the  other  hand,  one  might  argue  that  the  effect  attributed  to  ASTAMIDS 
might  be,  in  part,  a  learning  effect  because  the  ASTAMIDS  runs  usually  followed  (by 
about  two  days)  the  corresponding  runs  without  ASTAMIDS  and  minefields  were 
deployed  identically  in  corresponding  cases.  R^,  the  portion  of  the  variance  accounted  for 
by  regression,  in  the  above  tables  are  0.1 1, 0.6,  and  0.25,  respectively. 

F.  SCENARIO  AREAS  (REPLICATIONS) 

Four  different  scenarios  (taking  place  on  four  different  areas  of  terrain)  were  used 
in  these  trials.  Each  involved  the  same  number  of  Blue  and  OPFOR  units,  but  differed 
somewhat  with  respect  to  the  number  of  minefields.  On  average  Area  1  contained  4.25 
fields.  Area  2  contained  4,  Area  3  contained  5,  and  Area  4  contained  4.25.  Every 
minefield  was  200  m  by  100  m  and  each  contained  140  AT  mines  and  80  AP  mines. 
Blue  forces  traveled  about  the  same  distance  in  all  cases;  but  some  variation  in  terrain 
was  apparent  from  maps  of  the  four  areas.  When  trials  were  grouped  by  these  geographic 
areas,  they  differed  with  respect  to  outcomes. 

Figure  8  suggests  a  strong  difference  between  outcomes  of  trials  taking  place  on 
Area  1  and  the  three  other  terrain  patches  in  CME.  Losses  in  all  categories  are  greater  on 
average  in  Area  1 .  Also,  there  is  some  uniformity  with  respect  to  losses  to  mines  among 
the  remaining  three,  while  losses  to  direct  and  indirect  fire  show  some  variation.  While  a 
definitive  explanation  of  why  Area  1  stands  out  may  prove  elusive,  some  simple 
conjectures  come  to  mind  quickly.  One  is  that  the  runs  were  conducted  in  order  with 
respect  to  areas.  Thus,  the  Blue  force’s  first  exposure  to  a  given  mix  of  countermine 
equipment  occurred  on  Area  1.  There  was  only  one  exception  to  this  practice,  which 
occurred  in  the  second  vignette  when  an  Area  1  trial  was  run  after  the  corresponding  trials 
on  Areas  2,  3,  and  4.  It  seems  reasonable  to  suspect  that  learning  may  account  for  the 
higher  losses  associated  with  Area  1.  (Recall  that  an  elementary  regression-based 
analysis  indicated  no  influence  of  the  overall  order  of  mns  on  the  number  of  Blue  losses.) 
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Figure  8.  Blue  Losses  Vs  Area  (Scenario) 

Another  conjecture  about  the  vulnerability  of  Blue  forces  in  Area  1  has  to  do  with 
the  “lay  of  the  land.”  In  Area  1,  Blue  was  required  to  traverse  a  corridor  (the  long  axis  of 
a  low-lying  basin)  in  order  to  secure  its  objective.  This  topographical  feature  may  have 
exposed  Blue  to  direct  fire  and  channeled  its  forces  in  some  way  that  made  them  less  able 
to  avoid  mines.  All  other  terrain  sections  were  planar  or  irregular  and  may  have  afforded 
Blue  more  lateral  movement.  In  any  case,  they  did  not  channel  the  Blue  force  to  the  same 
extent  as  Area  1 . 

Appendix  B  contains  the  sum  of  squares  due  to  replications  (scenario  areas)  and 
the  error  sum  of  squares  from  which  ANOVA  tables  for  factors  and  interactions  can  be 
computed  after  variations  due  to  replications  are  taken  into  account.  These  modified 
ANOVA  tables  reveal  that  aerial  surveillance  and  the  interaction  between  aerial 
surveillance  and  breaching  system  become  more  significant  with  respect  to  losses  to 
mines  and  all  agents  combined,  while  ground  surveillance  becomes  significant  with 
respect  to  losses  to  direct  and  indirect  fire  (in  the  sense  that  GSTAMIDS  increases 
losses).  Essentially  the  same  issue  is  addressed  here,  but  the  analysis  mechanism  is 
stepwise  regression  with  scenario  areas  introduced  as  covariables  to  the  basic  linear 
model. 

The  following  tables  contain  data  related  to  the  impact  of  scenarios  in  the  analysis 
of  the  of  countermine  systems.  Again,  stepwise  regression  was  used  to  fit  the  linear 
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model  in  Section  LE  with  covariates  representing  the  four  individual  areas,  the  resulting 
expression  for  the  expected  outcome  on  the  ‘m’th  area  (1,2,3,  or  4)  now  becomes 

E(yijknm)  =  u  +  ai  +  bj  +  gk  +  Sm  +  abij  +  bgjk  +  gaki  +  Wyk 

In  this  formulation,  Sm  is  not  constrained  by  the  summation  condition  that  applies  to  the 
other  variables. 


Table  13.  F-Statistics  for  Model  with  Scenario  Area 


Source  of  Variation 

F-  statistics 

Losses  To  All 
Agents 

To  DF+IF 

To  Mines 

Area  (Scenario) 

Aerial  Surveillance 

Ground  Surveillance 

Interact  (Breaching  System  -  Aerial) 

14.2 

15.8 

4.6 

5.4 

6.0 

3.53* 

5.0 

(Adjusted  R^) 

Durbin-Watson 

0.40  (0.36) 

0.40  (0.36) 

0.36  (0.29) 

2.4 

1.68 

2.9 

*  The  value  of  F  differs  from  that  given  below  the  ANOVA  Table  9  (3.38)  because  the  former  used  the  full 
linear  model,  whereas  here  stepwise  regression  reduced  the  model  to  a  constant  plus  two  coefficients. 


Scenario  area  is  clearly  an  important  factor  in  all  of  the  measures  addressed  in 
Table  13.  Aerial  surveillance  impacts  both  losses  to  mines  and  overall  losses,  whereas 
ground  surveillance  is  a  significant  factor  with  respect  to  losses  to  direct  and  indirect  fire. 
That  is,  while  the  use  of  ASTAMDDS  reduces  Blue  losses  to  mines  and  all  agents,  the  use 
of  GSTAMIDS  increases  losses  to  direct  and  indirect  fire.  Again,  the  increase  in  losses 
may  be  due  to  the  increased  exposure  of  Blue  forces  while  they  search  for  minefields  or 
gaps  between  fields.  The  GSTAMIDS  vehicle  was  a  casualty  in  only  one  of  the  eight 
trials  in  which  it  was  deployed,  so  the  increased  losses  are  not  entirely  due  to  the  presence 
of  an  extra  vehicle.  Quantitatively,  the  impact  of  GSTAMIDS  is  manifested  by  the 
negative  coefficient  corresponding  to  the  absence  of  a  ground  surveillance  system  in 
Table  14,  below. 


Table  14.  Regression  Coefficients  for  Model  with  Scenario  Areas 


Value  of  Regression  Term  or  Coefficient 

Losses  To  All 
Agents 

To  DF+IF 

To  Mines 

u  -  Constant 

Si  -  Area  1 

ao  “  Aerial  Surveillance 

go  -  Ground  Surveillance 

aboo  *  Interact  (Breaching  System  -  Aerial) 

2.75 

1.58 

1.17 

3.75 

2.29 

1.46 

1.0 

0.72 

-0.47 

0.66 
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G.  TACTICS 

In  order  to  investigate  the  impact  of  tactics,  it  was  necessary  to  first  identify  those 
trials  in  which  breaching  or  bypassing  took  place.  While  one  would  expect  this  to  be 
entirely  straightforward,  it  does  contain  an  element  of  ambiguity.  Most  trials  in  which 
breaching  occurred  contain  at  least  one  launch  of  either  AVLM  or  ESMB.  However,  as 
mentioned  earlier,  rollers  and  plows  were  deployed  in  lieu  of  ESMB  in  two  trials.  Also, 
in  one  trial  in  which  AVLM  was  launched  in  anticipation  of  a  breaching  operation,  a 
bypass  route  was  discovered  and  the  breaching  operation  was  abandoned.  In  another, 
after  AVLM  was  launched  and  breaching  initiated,  losses  mounted  so  quickly  that  Blue 
withdrew  from  the  field  and  never  achieved  the  objective. 

Nonetheless,  launches  and  breaching  attempts  are  highly  correlated.  Table  15  lists 
all  the  trials  and  describes  each  in  terms  of  whether  or  not  a  breaching  operation  was 
initiated.  Breach  attempts  are  shown  in  shaded  boxes,  while  trials  with  bypassing 
operations  are  shown  as  clear. 


Table  15.  Breaching  and  Bypassing 


AST 

AST 

AST 

AST 

area  1 

area  2 

areas 

area  4 

area  1 

area  2 

area  3 

area  4 

AVLM  (base  case) 

breach 

pass 

breach® 

breach 

pass 

pass 

iHSiB 

pass 

ESMB 

wmmm 

breach 

pass 

breach 

breach 

brbach 

AVLM-GST 

pass 

:breach 

pass 

breach' 

pass 

pass 

pass 

flweaib' 

ESMB-GST 

pass 

breach 

breach 

breach 

breach 

breach 

pass 

pass 

Blue  withdrew  after  launching  AVLM. 

Breach  operations  conducted  with  rollers  and  plows  only. 
Blue  finds  bypass  after  launching  AVLM. 


From  Table  15,  Blue  initiated  19  breach  and  13  bypass  operations.  Four  bypasses 
were  conducted  without  ASTAMIDS,  and  five  were  conducted  without  GSTAMIDS. 
Only  one  was  conducted  with  neither  surveillance  system.  On  the  other  hand,  bypassing 
was  initiated  nine  times  in  the  presence  of  ASTAMIDS,  eight  times  with  GSTAMIDS, 
and  five  times  with  both.  Thus,  roughly  speaking,  there  was  a  50%  empirical  probability 
of  bypassing  (12  occurrences  out  of  a  possible  24)  when  some  sort  of  surveillance  system 
was  available,  but  only  about  12%  when  there  was  none. 

A  simple  2x2  contingency  table  confirms  that  the  availability  of  aerial 
surveillance  information  is  a  significant  factor  in  the  choice  of  tactics.  Surprisingly,  the 
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availability  of  ground  surveillance  is  not  (at  least,  not  nearly  to  the  same  degree).  While 
the  count  differences  are  small,  there  may  be  a  basic  difference  between  the  way 
ASTAMIDS  and  GSTAMIDS  are  used  that  ties  the  aerial  system  more  closely  to 
bypassing.  ASTAMIDS  was  a  reconnaissance  tool  deployed  in  the  first  phase  to  select 
routes,  whereas  GSTAMIDS  may  have  been  used  more  to  locate  the  edge  of  minefields 
after  routes  had  been  partially  traversed. 

Table  16  contains  the  number  of  times  Blue  chose  (or  at  least  prepared  to)  to 
breach  and  the  number  of  times  Blue  chose  to  bypass  or  circumvent  minefields  depending 
on  whether  or  not  ASTAMIDS  data  were  available.  The  standard  Pearson’s  Chi-square 
statistic  equals  3.24,  which  is  significant  below  the  10%  level  (suggesting  availability  of 
aerial  surveillance  data  and  tactical  decisions  are  not  independent). 


Table  16.  Influence  of  Aerial  Surveillance  and  Breaching  Systems  on  Choice  of  Tactics 

(contingency  tables) 


No  Aerial  Surveillance  ASTAMIDS 


Breaching  Tactic 

12 

7 

Bypassing  Tactic 

4 

9 

AVLM 

ESMB 

Breaching  Tactic 

7 

12 

Bypassing  Tactic 

9 

4 

A  similar  table  for  AVLM  vs.  ESMB  has  the  identical  entries  and  therefore  the 
same  Chi-square,  indicating  that  that  type  of  explosive  breaching  system  may  have  had  a 
strong  influence  on  tactics.  Finally,  the  very  last  table  (below.  Table  17)  indicates  that 
there  is  very  little  impact  on  choice  of  tactics  due  to  the  availability  of  a  ground 
surveillance  system.  In  this  case,  the  Chi-square  is  1.17,  which  is  not  significant  even  at 
the  25%  level. 

Table  17.  Influence  of  Ground  Surveillance  System  on  Choice  of  Tactics 


No  Ground  Surveillance  GSTAMIDS 


Breaching  Tactic 

11 

8 

Bypassing  Tactic 

5 

8 
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These  results  were  confirmed  by  stepwise  regression,  which  indicated  that  the 
significant  factors  determining  tactics  were  aerial  surveillance  and  type  of  explosive 
breaching  system.  Bypassing  was  the  more  likely  tactic  when  AVLM  was  the  breacher 
and  ASTAMIDS  data  were  on  hand. 

The  remainder  of  this  section  discusses  Blue  losses  as  a  function  of  chosen  tactic. 
We  begin  with  Table  18,  which  lists  the  mean  losses  by  agent  for  breaching  and 
bypassing.  The  same  table  with  standard  errors  of  the  means  appears  in  Appendix  C 
(from  which  one  can  determine,  say,  that  the  difference  in  mean  losses  to  mines  is 
significant  below  the  10%  level). 


Table  18.  Blue  Losses  Versus  Countermine  System  and  Tactic 


Blue  Losses 

Breach 

Bypass 

By  Mines 

2.21 

0.54 

By  D.F.+  I.F. 

2.16 

2.15 

Total 

4.37 

2.69 

It  is  not  surprising  that  Table  18  shows  a  substantial  decrease  in  losses  to  mines  as 
tactics  shift  from  breaching  to  bypassing.  After  all,  bypassing  suggests  avoiding  mines  by 
either  circumventing  minefields  or  penetrating  through  gaps  between  fields.  Mine 
encounters  should  only  occur  when  field  positions  are  incorrectly  estimated  or  when 
vehicles  wander  from  designated  paths.  Of  the  13  bypass  cases,  3  contained  losses  to 
mines  (all  in  Area  1).  In  two  of  these,  three  vehicles  were  lost  in  each  run;  one  vehicle 
was  lost  in  the  third.  In  each  of  these  three  cases  some  form  of  surveillance  system  was 
available  (two  with  GSTAMIDS,  one  with  ASTAMIDS).  In  neither  case  were  both 
available. 

Mean  losses  to  all  agents  differ  substantially,  but  this  difference  is  due  to  the 
differences  in  losses  to  mines.  Somewhat  surprisingly,  the  mean  losses  to  direct  and 
indirect  fire  for  both  breaching  and  bypassing  are  almost  identical.  It  is  not  clear  why  this 
should  be  so,  but  the  explanation  may  involve  the  fact  that  Blue  forces  spend  a  good  deal 
of  effort  searching  for  gaps  between  fields  when  attempting  minefield  avoidance.  The 
resulting  exposure  to  enemy  fire  while  searching  may  be  comparable  to  the  exposure 
endured  while  breaching. 

The  impact  of  tactics  was  analyzed  by  using  breaching  and  bypassing  as 
covariables  to  the  standard  model.  That  is,  in  addition  to  the  nine  terms  used  in  the 
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regression  in  Section  n.F,  a  tenth  term  was  added  to  account  for  tactics.  Thus  the 
resulting  model  becomes 

E(yijkmnp)  =  u  +  ai  +  bj  +  gk  +  Sm  +  tn  +  abjj  +  bgjk  +  gajd  +  Wjjk 

where  to  represents  breaching  and  ti  represents  bypassing. 

The  following  abbreviated  table  shows  the  results  of  a  stepwise  regression  in 
which  tactic,  along  with  the  countermine  systems  and  their  interactions,  is  a  variable.  A 
standard  10%  level  of  significance  is  used  as  an  entry  criterion,  so  that  all  variables 
represented  are  significant  at  that  or  a  lower  level. 


Table  19.  F*Statistics  with  Tactics,  Techniques,  and  Procedures  (TTP)  +  Area  Model 


F-  Statistics 

Source  of  Variation 

Losses  To  All 
Agents 

To  DF+IF 

To  Mines 

Tactic 

13.8 

24.5 

Area  (Scenario) 

25.3 

15.8 

15.6 

Ground  Surveillance 

3.53® 

Breaching  System 

3.36 

9.16 

Interact  (Breaching  System  -  Aerial) 

1  6.06 

(Adjusted  R^) 

0.53  (0.48) 

0.633  (0.359) 

0.61  (0.55) 

Durbin- Watson 

2.16 

1.68 

2.66 

The  value  of  F  differs  from  that  given  below  the  ANOVA  Table  9  (3.38)  because  the  former  used  the  full 
linear  model,  whereas  here  stepwise  regression  reduced  the  model  to  a  constant  plus  two  coefficients. 


Table  19  suggests  that  tactics  and  terrain  are  the  most  important  considerations 
with  respect  to  losses  to  mines  and  all  agents  combined.  Also,  as  in  the  case  of  the 
previous  model,  which  only  considered  variations  due  to  replications  (area),  terrain  and 
ground  surveillance  had  significant  impact  on  losses  to  direct  and  indirect  fire.  The 
greatest  surprise  in  Table  18  is  the  disappearance  of  aerial  surveillance/reconnaissance  as 
a  significant  factor  with  respect  to  losses  to  mines  and  all  agents.  This  is  most  likely  due 
to  the  fact  that  bypassing  and  the  availability  of  ASTAMIDS  data  are  so  closely  related. 
More  precisely,  although  aerial  surveillance  is  highly  correlated  with  losses  to  mines 
(correlation  coefficient  is  0.37,  which  is  significant  at  about  the  3.6%  level),  tactics  is 
even  more  highly  correlated  (0.42,  significant  at  the  1.6%  level).  More  importantly, 
however,  when  one  controls  for  tactics  (in  effect,  determine  correlation  when  tactics  are 
held  fixed),  the  partial  correlation  of  aerial  surveillance  with  losses  to  mines  drops  to 
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0.28,  which  is  only  significant  at  the  13.4%  level.  On  the  other  hand,  when  one  controls 
for  aerial  surveillance,  tactics  and  losses  to  mines  remain  significantly  correlated  (0.35, 
significant  at  5.5%).  This  is  essentially  the  process  through  which  stepwise  regression 
computes  the  most  relevant  factors.  A  similar  argument  holds  for  losses  to  all  agents 
combined. 

The  appearance  of  breaching  system  as  a  significant  factor  is  not  terribly 
surprising;  recall.  Blue  losses  averaged  about  0.5  vehicles  higher  with  AVLM  than  with 
ESMB,  overall,  and  0.7  with  respect  to  mines.  The  interaction  between  aerial 
surveillance  and  breaching  system  remains  significant,  however.  The  same  cautionary 
note  expressed  in  Section  n.D  regarding  interactions  and  main  effects  must  be  repeated 
here.  The  fact  that  significant  interaction  exists  between  the  level  of  aerial  surveillance 
and  breaching  system  (with  respect  to  losses  to  mines)  implies  that  the  impact  of 
breaching  systems  depends  on  the  level  of  aerial  surveillance  and  cannot  be  estimated 
without  imposing  the  constraint  that  interactions  sum  to  zero  (over  each  index).  The 
extent  to  which  the  reader  can  accept  this  constraint  has  bearing  on  the  extent  to  which  he 
or  she  is  willing  to  accept  the  conclusion. 

Table  20  contains  the  regression  coefficients  for  each  of  the  significant  terms  for 
each  of  the  measures:  losses  to  all  agents,  direct  and  indirect  fire,  and  mines.  Again,  as  in 
Section  n.D,  a  zero  subscript  connotes  the  lower  level  of  the  system  (e.g.,  ‘go’means 
GSTAMINDS  is  not  used  and  ‘bo’  implies  AVLM).  Recall,  the  symbol  ‘to’  denotes  the 
breaching  tactic.  Positive  coefficients  indicate  an  increase  in  losses.  Again,  the 
appearance  of  a  negative  coefficient  for  ground  surveillance  with  respect  to  losses  to 
direct  and  indirect  fire  indicates  that  these  losses  are  reduced  without  GSTAMn)S. 


Table  20.  Stepwise  Regression  Constants  and  Coefficients  with  TTP  +  Area  Modei 


Value  of  Regression  Term  or  Coefficient 

All  Agents 

DF  +  IF 

Mines 

u-  Constant 

2.2 

1.58 

0.74 

to  -  Tactic 

1.6 

1.29 

Si  -  Area 

4.7 

2.3 

2.2 

bo  -  Breaching  System 

0.75 

0.74 

go  "  Ground  Surveillance 

1  -0.47 

aboo  ‘  Interact  (Aerial  -  Breaching  System) 

0.58 
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III.  CONCLUSIONS  AND  RECOMMENDATIONS 


Three  points  stand  out  from  the  analyses  of  CME  data.  First,  having  ASTAMIDS 
intelligence  reduced  losses  due  to  mines  and  all  agents  combined.  This  was  due,  at  least 
in  part,  to  the  fact  that  bypassing  as  a  tactic  was  employed  at  a  relatively  high  frequency 
when  ASTAMIDS  data  were  available. 

Secondly,  the  use  of  GSTAMIDS  is  costly  with  respect  to  losses  from  direct  and 
indirect  fire,  especially  in  breaching  operations.  Unsurprisingly,  Blue’s  losses  to  mines 
can  be  reduced  by  circumventing  or  penetrating  gaps  between  minefields  without  any 
apparent  increase  in  losses  to  direct  or  indirect  fire. 

Finally,  there  appears  to  be  a  strong  interaction — with  respect  to  Blue  losses  to 
mines — ^between  aerial  surveillance  and  the  explosive  breaching  system.  This  is  likely  to 
be  due  to  the  fact  that  bypassing  was  the  preferred  tactic  when  AVLM  and  ASTAMIDS 
data  were  available.  The  possibility  of  a  tendency  (perhaps  ascribable  to  the  desire  to 
exercise  a  new  system)  to  breach  when  ESMB  was  on  hand  should  not  be  discounted. 
When  tactics  are  taken  into  consideration,  ESMB  reduces  losses  to  mines,  however. 

These  issues  should  be  investigated  in  greater  detail.  An  appropriate  research 
mechanism  might  be  a  constructive  model  in  which  large  numbers  of  trials  can  be 
generated.  It  is  highly  recommended  that  future  test  designs  include  tactics  (breaching 
versus  bypassing)  as  a  principal  factor. 


37 


UNCLASSIFIED 


UNCLASSIFIED 


(This  page  is  intentionally  blank.) 


38 


UNCLASSIFIED 


UNCLASSIFIED 


APPENDIX  A 

BLUE  LOSSES  AND  PARTIAL  KILLER-VICTIM  SCOREBOARDS 
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Appendix  A 

BLUE  LOSSES  AND  PARTIAL  KILLER-VICTIM  SCOREBOARDS 


Table  A-1.  Blue  Losses  to  Mines 


No  ASTAMIDS 

ASTAMIDS 

area  1 

area  2 

area  3 

area  4 

area  1 

area  2 

areas 

area  4 

AVLM 

8 

I  0 

6 

I  1 

3 

0 

0 

I  0 

ESMB 

2 

I  1 

3 

i  1 

0 

2 

0 

I  2 

AVLM-GSTAMIDS 

3 

I  ^ 

0 

I  4 

0 

0 

0 

I  1 

ESMB-GSTAMIDS 

1 

I  1 

0 

I  1 

4 

1 

0 

0 

Table  A-2.  Blue  Losses  to  Direct  and  Indirect  Fire 


No  ASTAMIDS 

ASTAMIDS 

area  1 

area  2 

area  3 

area  4 

area  1 

area  2 

areas 

area  4 

AVLM 

4 

0  j 

2 

0 

5  i 

1 

I  ^  i 

0 

ESMB 

2 

I  0  I 

2 

3 

4  I 

0 

I  ^ 

2 

AVLM-GSTAMIDS 

5 

i  1  ! 

2 

3 

3  I 

3 

I  1  i 

2 

ESMB-GSTAMIDS 

3 

I  2  I 

3 

7 

5  I 

1 

I  1  I 

0 

Tabie  A-3.  Data  for  Figure  1 


Kills 

AVLM 

AVLM- 

AST 

ESMB 

ESMB-AST 

AVLM-GST 

AVLM-GST- 

AST 

ESMB- 

GST 

ESMB-GST- 

AST 

By  Mines 

3.75 

0.75 

1.75 

1 

2.75 

0.25 

0.75 

1.25 

By  D.F. 

1.5 

1.5 

1.75 

1.25 

2 

0.75 

2.5 

1.25 

By  I.F 

0 

0.25 

0 

0.5 

0.75 

1.5 

1.25 

0.5 

Total 

5.25 

2.5 

3.5 

2.75 

5.5 

2.5 

4.5 

3 

Losses  were  computed  by  counting  all  vehicles  that  had  at  least  one  of  the 
standard  damage  levels:  mobility,  fire  power,  or  catastrophic.  For  vehicles  suffering  only 
mobility  and  fire-power  kills,  credit  was  ascribed  to  the  agent  inflicting  the  first  kill.  In 
the  case  of  vehicles  suffering  mobility  or  firepower  followed  by  a  subsequent  catastrophic 
kill,  the  agent  inflicting  the  catastrophic  kill  was  given  credit. 
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The  following  table  is  an  abbreviated  killer-victim  scoreboard.  All  Blue  vehicles 
are  represented,  but  not  in  all  possible  subcategories.  Categories  presented  are;  Manned 
Armor  Vehicles,  which  are  the  MlAls  containing  live  tank  crews;  simulated  ModSAF 
MlAls,  referred  to  as  the  Simulated  Armor  Vehicles;  MlAls  with  rollers  and  plows, 
which  were  simulated  in  ModSAF;  Blue  countermine  and  engineering  vehicles,  both 
manned  and  simualted. 


Table  A-4.  Killer  -  Victim  Scoreboard 


AVLM 

AVLM-AST 

ESMB 

ESMB-AST 

AVLM-GST 

AVLM- 

GST-AST 

ESMB- 

GST 

ESMB- 

GST-AST 

Blue  Armor  (Manned  M1  Als) 

by  mines 

1 

0.5 

0.75 

0 

0 

0 

0 

0.75 

byDF 

0 

0 

0 

0 

0 

0 

0 

0 

by  IF 

0 

0 

0 

0 

0 

0 

0 

0 

Blue  Armor  (Simulated  M1A1s-  without  plows  and  rollers) 

by  mines 

1 

0 

0 

0 

1 

0 

0.5 

0 

by  DF 

0.75 

0.5 

1.75 

0.5 

0.5 

0.5 

2 

1 

by  IF 

0 

0.25 

0 

0.25 

0.25 

0 

0.5 

0.25 

Blue  Armor  (Simulated  M1 A1-  with  plows  and  rollers) 

by  mines 

0.75 

0.25 

0.5 

0.75 

1 

0 

0.25 

0.5 

byDF 

0.75 

1 

0 

0.75 

1 

0.25 

0.5 

0.25 

by  IF 

0 

0 

0 

0 

0.25 

0.25 

0.25 

0.25 

Blue  Countermine  and  Engineering  Vehicles  (Manned  and  Simulated) 

by  mines 

1 

0 

0.5 

0.25 

0.75 

0.25 

0 

0 

byDF 

0 

0 

0 

0 

0.5 

0 

0 

0 

by  IF 

0 

0 

0 

0.25 

0.25 

1.25 

0.5 

0 

Total 

5.25 

2.5 

3.5 

2.75 

5.5 

2.5 

4.5 

3 

Mines 

3.75 

0.75 

1.75 

1 

2.75 

0.25 

0.75 

1.25 

DF 

1.5 

1.5 

1.75 

1.25 

2 

0.75 

2.5 

1.25 

IF 

0 

0.25 

0 

0.5 

0.75 

1.5 

1.25 

0.5 

A-2 


UNCLASSIFIED 


UNCLASSIFIED 


APPENDIX  B 
ANOVA  TABLES 


UNCLASSIFIED 


UNCLASSIFIED 


APPENDIX  B 
ANOVA  TABLES 


Table  B-1.  ANOVA  for  Blue  Losses  to  All  Agents 


Source  of  Variation 

Sum  of  Squares 

D.F. 

Mean 

F 

Breaching  System 

2 

1 

2 

0.2 

Ground  Surveillance 

1,125 

1 

1.125 

0.11 

Aerial  Surveillance 

32 

1 

32 

3.123 

Interact  (breacher-ground) 

0.5 

1 

0.5 

0.05 

Interact  (breacher-aerial) 

6.125 

1 

6.125 

0.6 

Interact  (ground-aerial) 

0.5 

1 

0.5 

0.05 

Interact  (all) 

0.125 

1 

0.125 

0.012 

Error 

246.5 

24 

10.27 

Total 

288.875 

31 

3  Significant  at  10%  level;  becomes  4.31  when  variance  due  to  replications  (areas)  is  taken  into  account. 
(The  sum  of  squares  due  to  replications  is  90.6,  with  3  degrees  of  freedom.) 


Table  B-2.  ANOVA  for  Blue  Losses  to  Mines 


Source  of  Variation 

Sum  of  Squares 

D.F. 

Mean 

F 

Breaching  system 

3.78 

1 

3,78 

1.12 

Ground  surveillance 

2.53 

1 

2.53 

0.75 

Aerial  surveillance 

16.53 

1 

16,53 

4.883 

Interact  (breacher-ground) 

0.28 

1 

0.28 

0.08 

Interact  (breacher-aehal) 

13.78 

1 

13.78 

4.07b 

Interact  (ground-aerial) 

1.53 

1 

1.53 

0.45 

Interact  (all) 

0.28 

1 

0.28 

0.83 

Error 

81.25 

24 

3.385 

Total 

119.97 

31 

3  Significant  at  5%  level;  becomes  5.08  when  variance  due  to  replications  is  taken  into  account, 
b  Significant  at  10%  level;  becomes  4.23  when  variance  due  to  replications  is  taken  into  account. 
(The  sum  of  squares  due  to  replications  is  12.8,  with  3  degrees  of  freedom.) 
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Table  B-3.  ANOVA  for  Blue  Losses  to  Direct  and  Indirect  Fire 


Source  of  Variation 

Sum  of  Squares 

D.F. 

Mean 

F 

Breaching  System 

0.28 

1 

0.28 

0.084 

Ground  Surveillance 

7.03 

1 

7.03 

2.ia 

Aerial  Surveillance 

2.53 

1 

2.53 

0.76 

Interact  (breacher-ground) 

0.31 

1 

0.31 

0.0093 

Interact  (breacher-aerial) 

1.53 

1 

1.53 

0.46 

Interact  (ground-aerial) 

3.78 

1 

3.78 

1.13b 

Interact  (all) 

0.78 

1 

0.78 

0.23 

Error 

80.25 

24 

3.34 

Total 

96.2 

31 

a  Becomes  3.38  when  variance  due  to  area  is  taken  into  account  (10%  level  of  significance), 
b  Becomes  1 .82  when  variance  due  to  area  is  taken  into  account. 

(The  sum  of  squares  due  to  replications  is  36.6,  with  3  degrees  of  freedom.) 
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Appendix  C 

PAIRWISE  COMPARISONS 


Table  C-1  summarizes  the  mean  number  of  kills  of  Blue  vehicles  by  OPFOR 
agent.  The  first  row  shows  the  technology  available  in  the  16  trials  over  which  the  kills 
were  averaged.  Pairwise  t-tests  show  significant  differences  (below  10%)  in  the 
following  cases:  losses  to  mines  with  and  without  ASTAMIDS,  overall  losses  with  and 
without  ASTAMIDS,  and  losses  to  direct  and  indirect  fire  with  and  without  GSTAMIDS. 


Table  C-1.  Mean  Blue  Losses  by  Agent  Versus  Countermine  System 
(with  standard  error  of  the  mean) 


Blue 

Losses 

AST 

No  AST 

AVLM 

ESMB 

GST 

No  GST 

By  Mines 

0.81  (0.32) 
1.88  (0.42) 
2.69  (0.65) 

2.25  (0.57) 
2.44  (0.47) 

4.69  (0.80) 

1.88  (0.63) 
2.06  (0.41) 
3.94  (0.91) 

1.19(0.29) 
2.25  (0.48) 
3.44(0.61) 

1.25  (0.39) 
2.63  (0.46) 
3.88  (0.69) 

1.81  (0.58) 
1.69  (0.41) 

3.5  (0.85) 

By  D.F.+  I.F 

Total 

Table  C-2  is  a  copy  of  Table  17  in  the  text  with  standard  errors  of  the  mean 
included  for  completeness. 


Table  C-2.  Blue  Losses  Versus  Countermine  System  and  Tactic 
(mean  and  standard  error  of  mean) 


Blue  Losses 

Breach 

Bypass 

By  Mines 

2.21  (0.49) 
2.16(0.41) 

4.37  (0.72) 

0.54(0.31) 

2.15(0.50) 

2.69  (0.76) 

By  D.F.+  I.F. 

Total 
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